Skip to content

feat: Allow servers to express supported endpoints with ConfigResponse #2848

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
kevinjqliu merged 3 commits into from
Dec 29, 2025
Merged

feat: Allow servers to express supported endpoints with ConfigResponse #2848

kevinjqliu merged 3 commits into from
Dec 29, 2025

Conversation

@geruh
Copy link
Contributor

@geruh geruh commented Dec 20, 2025

closes to #2847

Rationale for this change

This PR adds the server endpoint capabilities support, aligning with the Java implementation. While working on the REST scanning support, we need to know if a server supports specific capabilities before making any calls. So this PR also adds some extra support for the current implementation of PI iceberg REST catalog.

The REST catalog will now parse the endpoints field from the config call to determine server capabilities. When a server doesn't respond, we have fallback logic that matches the behavior of Java's rest catalog. The View endpoints are conditionally added to the default with the config property as well.

Are these changes tested?

Added unit tests and tested with the iceberg rest fixture.

Are there any user-facing changes?

Yes added config and alignment with java impl.

cc: @kevinjqliu @Fokko

@kevinjqliu
Copy link
Contributor

kevinjqliu commented Dec 24, 2025

wdyt about adding integration tests against the iceberg-rest-fixture?

Running the integration test infra gives me this response on http://localhost:8181/v1/config

{
  "defaults": {},
  "overrides": { "namespace-separator": "%2E" },
  "endpoints":
    [
      "POST v1/oauth/tokens",
      "POST https://auth-server.com/token",
      "GET v1/config",
      "GET /v1/{prefix}/namespaces",
      "POST /v1/{prefix}/namespaces",
      "HEAD /v1/{prefix}/namespaces/{namespace}",
      "GET /v1/{prefix}/namespaces/{namespace}",
      "DELETE /v1/{prefix}/namespaces/{namespace}",
      "POST /v1/{prefix}/namespaces/{namespace}/properties",
      "GET /v1/{prefix}/namespaces/{namespace}/tables",
      "POST /v1/{prefix}/namespaces/{namespace}/tables",
      "HEAD /v1/{prefix}/namespaces/{namespace}/tables/{table}",
      "GET /v1/{prefix}/namespaces/{namespace}/tables/{table}",
      "POST /v1/{prefix}/namespaces/{namespace}/register",
      "POST /v1/{prefix}/namespaces/{namespace}/tables/{table}",
      "DELETE /v1/{prefix}/namespaces/{namespace}/tables/{table}",
      "POST /v1/{prefix}/tables/rename",
      "POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/metrics",
      "POST /v1/{prefix}/transactions/commit",
      "GET /v1/{prefix}/namespaces/{namespace}/views",
      "HEAD /v1/{prefix}/namespaces/{namespace}/views/{view}",
      "GET /v1/{prefix}/namespaces/{namespace}/views/{view}",
      "POST /v1/{prefix}/namespaces/{namespace}/views",
      "POST /v1/{prefix}/namespaces/{namespace}/views/{view}",
      "POST /v1/{prefix}/views/rename",
      "DELETE /v1/{prefix}/namespaces/{namespace}/views/{view}",
      "POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/plan",
      "GET /v1/{prefix}/namespaces/{namespace}/tables/{table}/plan/{plan-id}",
      "POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/tasks",
      "DELETE /v1/{prefix}/namespaces/{namespace}/tables/{table}/plan/{plan-id}",
    ],
}

kevinjqliu
kevinjqliu approved these changes Dec 24, 2025
View reviewed changes
Copy link
Contributor

@kevinjqliu kevinjqliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for adding this feature.
The PR looks good and is throughly tested. I just have a few nit comments.
Feel free to address here or as a follow up PR

@geruh geruh mentioned this pull request Dec 25, 2025
Merged
kevinjqliu
kevinjqliu approved these changes Dec 29, 2025
View reviewed changes
Copy link
Contributor

@kevinjqliu kevinjqliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! One more comment about tests

We should probably test against the IRC server for integration test, maybe in tests/integration/test_rest_catalog.py. Can add to this PR or as a follow up

@kevinjqliu kevinjqliu requested a review from Fokko December 29, 2025 22:44
@kevinjqliu kevinjqliu merged commit a8033c1 into apache:main Dec 29, 2025
8 checks passed
@kevinjqliu
Copy link
Contributor

kevinjqliu commented Dec 29, 2025

Thanks!

@kevinjqliu kevinjqliu mentioned this pull request Jan 9, 2026
Closed
Fokko pushed a commit that referenced this pull request Jan 16, 2026
@geruh
feat: Add support for rest scan planning (#2864)
28fabdd 
related to #2775 

# Rationale for this change

Adds **synchornous** client-side support for REST server side scan
planning, allowing for scanning if the rest catalog supports it.

This PR cherry-picks and builds on two WIP PRs:
  - Rest Models #2861  
  - Endpoints PR #2848 

Currently scanning is enable with rest-scan-planning-enabled=true in
catalog properties.

TODO: spec handling 

## Are these changes tested?

Integration tests added with manual testing

## Are there any user-facing changes?

yes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kevinjqliu kevinjqliu kevinjqliu approved these changes

@Fokko Fokko Awaiting requested review from Fokko

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@geruh @kevinjqliu